Efficient code cache management in presence of infrequently used complied code fragments

ABSTRACT

A process, a computer program product, and a computer system for efficient code cache management implemented by a just-in-time compiler are provided. Embodiments of the present invention provide a practical approach to compile effectively for methods that are frequently executed over a short time period but are subsequently not executed frequently. The disclosed approach reduces the detrimental effects of compilation of such infrequently executed methods on throughput and garbage collection pause times.

TECHNICAL FIELD OF THE INVENTION

The present invention relates generally to programming languages andcompilers, and more particularly to efficient code cache management inthe presence of infrequently used compiled code fragments.

BACKGROUND

In enterprise class Java applications, such as WebSphere Commerce, theexecution profile typically includes a large number of methods; each ofthe methods takes a very small proportion of overall execution time.However, over time, these methods get compiled simply because theinitial compilation is driven by method invocation counts. Once a methodis executed certain times, the method gets compiled regardless of howlong it has taken to execute the method for the certain times. Thismeans that the Just-In-Time (JIT) compiler's code cache can contain manymethods that are executed only infrequently. Such compilations canincrease (code memory) footprints and startup/ramp-up time with littlebeneficial effect on throughput. However, the real problem stems fromthe fact that there is only a fixed amount of a code cache available foreach instance of the JIT compiler. If the code cache gets filled withmethods less important for throughput performance, more importantsubsequent compilations may not be successful since there is no space towrite the generated code. Interpreting the more frequently executedmethods which can not be compiled has a noticeable negative impact onthroughput, because interpreting is an order of magnitude (e.g., 10×)slower than compiled code.

The simplest approach to a bounded code cache is to flush its entirecontent when the bounded code cache becomes full. This solution,referred to as code pitching, is employed by Microsoft's .NET CompactFramework as described by Anthony, Leung, and Srisa-an (“To JIT or notto JIT: The Effect of Code Pitching on the Performance of .NETFramework”, 2005). As shown by the authors, while code pitching candegrade runtime performance, if it is done rarely enough, the penalty isbarely visible. Thus, picking an appropriate maximum size for the codecache is important. One drawback of the study, as admitted by theauthors, is that they considered only tiny applications which are notrepresentative of a real-life workload. For big applications, wherecompilation overhead is significant, simply throwing away all thecompiled code is going to put unnecessary strains on the computationalresources.

Zhang and Krintz (“Profile-Driven Code Unloading forResource-Constrained JVMs”, 2004) propose to periodically (e.g., every10 ms) discard, from the code cache, compiled methods which are unlikelyto be used in the future. The candidates for eviction are chosen withthe help of a lightweight profiling mechanism based on sampling. In afollow-up work (“Adaptive Code Unloading for Resource-Constrained JVMs”,2004), the authors develop an adaptive mechanism which determines whenmethod eviction should take place. The idea is that frequency of theeviction should be dictated by memory pressure: the more time the JVMspends in the garbage collector, the more frequent the eviction processshould be. However, this conclusion is a direct consequence of theirparticular implementation: code fragments reside in the Java heap andtherefore compete for space with Java objects. Freeing more code makesroom for Java objects and improves the time spent in the garbagecollector. An important disadvantage of the selective method evictionprocess is that it creates a lot of fragmentation.

Hazelwood and Smith (“Code Cache Management Schemes for DynamicOptimizers”, 2002) avoid fragmentation by evicting code fragments in aFIFO (First In First Out) manner. The code cache is considered a bigcircular buffer and eviction takes place on demand, when a new codefragment needs to be stored. While this approach simplifies code cachemanagement, it brings in the previous problem: because it is obliviousto the frequency of execution of code fragments, important compiledpieces of code can be evicted from the cache. However, the negativeeffects are likely milder than in the code pitching case where theentire cache is cleared.

Hazelwood and Smith (“Generational Cache Management of Code Traces inDynamic Optimization Systems”, 2003) propose the use of generationalcode caches. The code cache space is split into three regions: nursery,probation cache, and persistent cache. New code fragments are stored inthe nursery cache, possibly evicting other code fragments in case thereis not enough room. The victim of an eviction is moved into theprobation cache whose main role is to determine how important theevicted code fragments are. When the room needs to be made into theprobation cache, either the code fragment to be eliminated is moved intothe persistent code cache if it has been used since moved to theprobation cache, or the code fragment to be eliminated is simplydiscarded. This technique works relatively well, but complicates codecache management a great deal because of the following factors. (1)Moving code fragments around from one code cache to another requires thecode to be relocatable. (2) Direct transition from one code fragment toanother is not possible, unless metadata is kept around to fix all“callers” when the “callee” is moved or discarded (it is called “linkrepair” by the authors of the paper). (3) A mechanism that monitors codeusage is required to determine whether or not a piece of code should bemoved to the persistent code cache or discarded.

Throughput and garbage collection (GC) pause times can also beindirectly affected by the caching of a large number of infrequentlyused compiled methods. A GC cycle is often the perfect time to reclaimthe code cache space for unloaded or recompiled methods; the reclamationprocess includes locating and freeing up the run time metadata (e.g.,class hierarchy assumptions, GC relevant stack/registers information,exception tables, etc.). Different kinds of JIT metadata are stored indifferent hashtables (for example, in IBM J9 VM or IBM Testarossa JITcompiler) and hence locating all the information can be a very timeconsuming process that scales linearly with the amount of generatedcode. Even if the compilations are only infrequently used, the amount ofmetadata generated for these compilations can still affect the overalltime for locating JIT metadata.

SUMMARY

In one aspect, a process for efficient code cache management implementedby a just-in-time compiler is provided. The process includes:determining whether a method is frequently executed; placing a compiledcode of the method in a first code cache, in response to determiningthat the method is frequently executed; determining whether a secondcode cache is full, in response to determining that the method is notfrequently executed; placing the compiled code in the second code cache,in response to determining that the second code cache is not full; andflushing a subdivision of the second code cache and then placing thecompiled code in the second code cache, in response to determining thatthe second code cache is full.

In another aspect, a computer program product for efficient code cachemanagement implemented by a just-in-time compiler is provided. Thecomputer program product comprising a computer readable storage mediumhaving program code embodied therewith. The program code is executableto: determine whether a method is frequently executed; place a compiledcode of the method in a first code cache, in response to determiningthat the method is frequently executed; determine whether a second codecache is full, in response to determining that the method is notfrequently executed; place the compiled code in the second code cache,in response to determining that the second code cache is not full; andflush a subdivision of the second code cache and then place the compiledcode in the second code cache, in response to determining that thesecond code cache is full.

In yet another aspect, a computer system for efficient code cachemanagement implemented by a just-in-time compiler is provided. Thecomputer system comprises one or more processors, one or more computerreadable tangible storage devices, and program instructions stored on atleast one of the one or more computer readable tangible storage devicesfor execution by at least one of the one or more processors. The programinstructions are executable to determine whether a method is frequentlyexecuted. The program instructions are executable to, in response todetermining that the method is frequently executed, place a compiledcode of the method in a first code cache. The program instructions areexecutable to, in response to determining that the method is notfrequently executed, determine whether a second code cache is full. Theprogram instructions are executable to, in response to determining thatthe second code cache is not full, place the compiled code in the secondcode cache. The program instructions are executable to, in response todetermining that the second code cache is full, flush a subdivision ofthe second code cache and then place the compiled code in the secondcode cache.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a flowchart illustrating operational steps for efficient codecache management in presence of infrequently used compiled codefragments, in accordance with one embodiment of the present invention.

FIG. 2 is a diagram illustrating components of a computer device hostinga programming compiler that implements the operational steps shown inFIG. 1, in accordance with one embodiment of the present invention.

DETAILED DESCRIPTION

There are different types of infrequently executed methods: (1) methodsthat are executed frequently over a brief time period but are neverexecuted afterward (in which the containing class gets unloaded); (2)methods that are executed frequently over a brief time period but rarelyexecute afterwards (in which, naturally, the containing class is notunloaded); and (3) methods that are never executed frequently over anytime period (or window) but are executed infrequently over the whole run(in which the containing class is not unloaded). Embodiments of thepresent invention provide a practical approach to compile effectivelyfor methods that are frequently executed over a short time period butare subsequently not executed frequently (especially the first and thesecond types of methods mentioned above). It is beneficial to compilethe first and the second types of methods; however, once a certain timeperiod passes, the compilations are rarely if ever used again. A goal ofthe disclosed approach is to reduce the detrimental effects ofcompilation of such infrequently executed methods on throughput and GC(garbage collection) pause times. Although the approach can also beapplied to the third type of methods, it may not be as beneficial sinceit may end up compiling a lot of methods repeatedly.

In the embodiments of the present invention, the approach includes aheuristic to identify methods that are likely to execute frequently overa short time period but are rarely executed afterwards. Identifyinginfrequently executed methods can be done using a static heuristic or adynamic heuristic. An example is using the static heuristics forcompilations from auto generated classes, i.e., classes that are notfrom .class files on the disk but generated on the fly by the runningJava program. Usually, the auto generated classes are associated with aclass loader that is instantiated for some task (e.g., handling acustomer session); after the task is completed, the class loader isgarbage-collected and all the classes loaded by the class loader areunloaded. Another example is using the static heuristics forcompilations from Java Server Pages (JSP) classes, i.e., methods inclasses produced by the JSP compiler. These are used fairly infrequentlyin practice, because the JSP compiler in WebSphere Application Server(WAS) produces many different methods that are different only slightlyin terms of constants hardcoded in the bytecodes. Dynamic heuristics canbe more reliable in terms of identifying infrequently executed methodsin a given workload, and additionally dynamic heuristics can also detectphase changes that lead to a class of methods no longer beinginfrequently executed. Regarding determining whether a method isfrequently executed, the program must monitor an invocation count of themethod. If the invocation count of the method exceeds a predeterminednumber, the method is deemed frequently executed. Note that any methodscompiled at higher optimization levels as a result of the JIT compiler'susual heuristics to determine hotness (based on sampling) should not beconsidered infrequently executed.

The approach further includes segregated code caches: N code caches forinfrequently executed methods (hereafter referred to as “transient codecache(s)”) and M code caches for everything else (hereafter referred toas “permanent code cache(s)”). Without loss of generality, N=1 and M=1are assumed in the following discussion of embodiments. In embodimentsof this invention, a code cache space is logically split into tworegions with different responsibilities and functionality: a transientcode cache that stores infrequently executed methods and a permanentcode cache that stores frequently executed methods. The choice for theproportion of the overall code cache to use for the infrequentlyexecuted methods is implementation dependent. The exact size can bechosen once at JVM (Java Virtual Machine) startup; alternatively, theexact size can be varied as the program runs (i.e., based on the dynamicbehavior observed), subject to availability of the code cache space. TheJIT compiler can also choose to subdivide the space available forfrequently and infrequently executed methods compilations into severaldistinct code caches using standard heuristics, e.g., typically the JITcompiler employs multiple compilation threads to do compilations andeach thread has a dedicated code cache to avoid synchronization issueswhen emitting code. In addition to segregating the compiled code, theapproach includes segregating the JIT metadata associated with thecompiled code for infrequently executed methods into a separate toplevel structure. Segregating the JIT metadata has efficiency benefitswhen the code cache is to be flushed. The JIT metadata for all methodsfrom the transient code cache are stored in a small number of dedicated“bulk allocated” memory segments. This arrangement enables a relativelycheap way to free up memory when the transient code cache is to beflushed. Instead of performing a linear walk over the metadataassociated with the code in the transient code cache and freeing themindividually (or piece by piece), the JIT compiler frees the dedicatedsegments in bulk. Moreover, by segregating the JIT metadata for thoseinfrequently executed methods, they will not be encountered when the JITmetadata structure is walked for other more frequently executed methods;therefore, there is a small “execution speed” benefit to segregating themetadata as well.

The approach yet further includes a mechanism to deal with the transientcode cache that is full: to flush all the code in one or more ofsubdivisions of the transient code cache and to delete all the JIT(Just-in-Time) metadata for methods in the one or more of thesubdivisions. Flushing is only possible by preventing any futureinvocations of methods compiled in the code cache and transferring outof any invocations already in progress by using the general OSR (OnStack Replacement) mechanism. The embodiments provide a simple yetpractical scheme to reduce the compensation cost associated with gettingrid of the code for methods in the transient code cache, without havingto walk code locations and patch them in any way that is possible,specifically because the methods in the transient code cache are chosensuch that they are infrequently executed. Flushing the one or more ofthe subdivisions of the transient code cache and discarding all thegenerated code therein reverse the state of methods in the one or moreof the subdivisions of the transient code cache to interpreted code.This in turn resets the invocation count to a non-negative value andrestarts to count the invocation of the methods. The non-negative valuefor a given method depends on the policy for the next compilation, i.e.,to accelerate the next compilation.

Generally speaking, dispatches to methods can be done via direct callsor indirect calls (e.g., via a virtual function table or interfacetable). Even when the source code contains an indirect call, the JITcompiler may employ an approach such as PIC (polymorphic inline cache)to emit a guard test followed by a direct call to the methods calledfrom a given call site. For this reason, the proportion of direct callsfound in a compiled body can be quite high. Because the broken caller tocallee links need to be repaired, the direct calls are problematic forthe flushing of the transient code cache. This problem is typicallyresolved by maintaining some metadata around; the metadata allows theidentification of all call sites which need to be patched. However, theapproach in this invention does not follow the typical solution. In theembodiments of the present invention, for a caller/callee AB pair, wheremethod A does not belong to the transient code cache while method Bbelongs to the transient cache, the JIT compiler is not allowed togenerate a direct call. Instead, the JIT compiler makes the call gothrough some small amount of general “glue” instructions, which willlook up the target address from a field on the JVM's method datastructure and do the dispatch. The main advantage of the approach inthis invention is that the JIT compiler only needs to patch a smallamount of memory in the JVM's method data structure andvirtual/interface method tables, without keeping track of code memorylocations. This approach does not impose a performance penalty becausethe methods in the transient code cache are supposed to be called veryinfrequently. On the other hand, if both the caller and the calleereside in the transient code cache, direct calls from one to another areperfectly feasible since both methods disappear at the same time duringthe flush operation.

The approach yet further includes a backup mechanism, whereby theJVM/JIT compiler detects incorrect guesses for expected lifetimes ofmethods and incorrect placement of compiled code for such methods, andthe JVM/JIT compiler feeds this back into the heuristic so as to adaptto the actual behavior of the program. The backup mechanism is employedto detect whether the same method is repeatedly being compiled becausethe method is placed into the transient code cache and flushedrepeatedly. The JVM/JIT compiler can keep track of the number of timesthe method is compiled into the transient code cache, via a simplecounter in the JVM's representation of the method. Once the method iscompiled a certain number of times into the transient code cache, theJVM/JIT compiler will compile it into the permanent code cache. At thesame time, the JVM/JIT compiler keeps track of how many times a specificmethod placement heuristic has yielded a wrong decision (placement inthe transient code cache, followed by a flush, followed by arecompilation) irrespective of the method identity. Once a certainthreshold of wrong decisions has been reached, the JVM/JIT alters thespecific heuristic that has identified that the method as a candidatefor the transient code cache (e.g., to turn the specific heuristic off).In other words, each decision about whether the compiled code of themethod is placed in the transient code cache is fed back to the overallheuristic to change the aggression level for identifying candidates inthe future.

The approach provided in embodiments of this invention has the followingadvantages. (1) It does not suffer fragmentation issues because theapproach flushes the transient code cache and does not leave compilationstubs behind. (2) Flushing is restricted to the transient cache suchthat important methods are unlikely to be evicted as long as theselection of heuristics works reasonably well. (3) Code fragments arenot moved from one code cache to another such that the generated codedoes not have to be relocatable. (4) No metadata is needed to deal withbroken links when code fragments are discarded. (5) GC pause times areimproved due to faster code cache management. (6) Searching for metadatain the mainline (non-transient) code cache is faster because theinformation regarding the infrequently used methods has been moved to aseparate data structure.

FIG. 1 is flowchart 100 illustrating operational steps for efficientcode cache management in presence of infrequently used compiled codefragments, in accordance with one embodiment of the present invention.At step 101, a programming compiler uses a heuristic for deciding methodplacement. At this step, the programming compiler applies the heuristicto identify methods that are executed frequently over a short timeperiod but are rarely executed afterwards. In the next steps, theprogramming compiler will use the heuristic to determine whether themethods are frequently executed and where the methods are placed. Inprevious paragraphs of this document, the heuristic has been discussedin detail.

Referring to FIG. 1, at decision block 103, the programming compilerusing the heuristic determines whether a method is frequently executed.In response to determining that the method is frequently executed (YESbranch of decision block 103), the programming compiler at step 117 putsa compiled code for the method in a permanent code cache and also putsmetadata of the compiled code for the method in the permanent codecache. As described in previous paragraphs of this document, thepermanent code cache is used to store frequently executed methods.

In response to determining that the method is not frequently executed(NO branch of decision block 103), at decision block 105, theprogramming compiler determines whether the number of recompilations dueto the heuristic exceeds a first predetermined value. The programmingcompiler keeps track of how many times the heuristic has yielded a wrongdecision.

In response to determining that the number of the recompilations due tothe heuristic exceeds the first predetermined value (YES branch ofdecision block 105), the programming compiler at step 107 adjusts theheuristic. For example, the programming compiler changes the aggressionlevel for identifying frequently executed methods.

In response to determining that the number of the recompilations due tothe heuristic does not exceed the first predetermined value (NO branchof decision block 105) or after step 107, the programming compiler atdecision block 109 determines whether the number of recompilations dueto the method exceeds a second predetermined value. Using the backupmechanism described in previous paragraph of this document, theprogramming compiler checks whether the same method is repeatedly beingcompiled because the method is placed into a transient code cache andflushed repeatedly.

In response to determining that the number of the recompilations due tothe method exceeds the second predetermined value (YES branch ofdecision block 109), the programming compiler executes step 117 (i.e.,puts a compiled code for the method in the permanent code cache and alsoputs metadata of the compiled code for the method in the permanent codecache).

In response to determining that the number of the recompilations due tothe method does not exceed the second predetermined value (NO branch ofdecision block 109), the programming compiler, at decision block 111,determines whether a transient code cache is full. The transient codecache is used to store infrequently executed methods.

In response to determining that the transient code cache is full (YESbranch of decision block 111), the programming compiler at step 113flushes a subdivision of the transient code cache and also flushesmetadata in the subdivision. At this step, the programming compilerflushes all the code in the subdivision of the transient code cache anddeletes all the JIT (Just-in-Time) metadata for methods in thesubdivisions.

In response to determining that the transient code cache is not full (NObranch of decision block 111) or after step 113, the programmingcompiler puts at step 115 the compiled code of the method in thetransient code cache and puts the metadata of the compiled code for themethod in the transient code cache.

FIG. 2 diagram illustrating components of computer device 200 hosting aprogramming compiler that implements the operational steps shown in FIG.1, in accordance with one embodiment of the present invention. It shouldbe appreciated that FIG. 2 provides only an illustration of oneimplementation and does not imply any limitations with regard to theenvironment in which different embodiments may be implemented.

Referring to FIG. 2, computer device 200 includes processor(s) 220,memory 210, tangible storage device(s) 230, network interface(s) 240,and I/O (input/output) interface(s) 250. In FIG. 2, communications amongthe above-mentioned components of computing device 200 are denoted bynumeral 290. Memory 210 includes ROM(s) (Read Only Memory) 211, RAM(s)(Random Access Memory) 213, and cache(s) 215. One or more operatingsystems 231 and one or more computer programs 233 reside on one or morecomputer readable tangible storage device(s) 230. The programmingcompiler or the JIT (Just-in-Time) compiler 235 resides on one or morecomputer readable tangible storage device(s) 230. Computing device 200further includes I/O interface(s) 250. I/O interface(s) 250 allows forinput and output of data with external device(s) 260 that may beconnected to computing device 200. Computing device 200 further includesnetwork interface(s) 240 for communications between computing device 200and a computer network.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device, such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network(LAN), a wide area network (WAN), and/or a wireless network. The networkmay comprise copper transmission cables, optical transmission fibers,wireless transmission, routers, firewalls, switches, gateway computersand/or edge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++, and conventionalprocedural programming languages, such as the “C” programming language,or similar programming languages. The computer readable programinstructions may execute entirely on the user's computer, partly on theuser's computer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer, or entirely on the remotecomputer or server. In the latter scenario, the remote computer may beconnected to the user's computer through any type of network, includinga local area network (LAN) or a wide area network (WAN), or theconnection may be made to an external computer (for example, through theInternet using an Internet Service Provider). In some embodiments,electronic circuitry including, for example, programmable logiccircuitry, field-programmable gate arrays (FPGA), or programmable logicarrays (PLA) may execute the computer readable program instructions byutilizing state information of the computer readable programinstructions to personalize the electronic circuitry in order to performaspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture, including instructions which implement aspectsof the function/act specified in the flowchart and/or block diagramblock or blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus, or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

What is claimed is:
 1. A process for efficient code cache managementimplemented by a just-in-time compiler, the process comprising: acomputer processor determining whether a method is frequently executed;in response to determining that the method is frequently executed, thecomputer processor placing a compiled code of the method in a first codecache which is a permanent code cache; the computer processordetermining whether a number of recompilations due to the method exceedsa predetermined number; in response to determining that the method isnot frequently executed and determining that the number ofrecompilations due to the method does not exceed the predeterminednumber, the computer processor determining whether a second code cacheis full, wherein the second code cache is a transient code cache; inresponse to determining that the second code cache is not full, thecomputer processor placing the compiled code in the second code cache;and in response to determining that the second code cache is full, thecomputer processor flushing a subdivision of the second code cache andthen placing the compiled code in the second code cache.
 2. The processof claim 1, further comprising: in response to determining that themethod is frequently executed, the computer processor placing metadataof the compiled code in the first code cache.
 3. The process of claim 1,further comprising: in response to determining that the method is notfrequently executed and the second code cache is full, the computerprocessor flushing metadata in the subdivision of the second code cache.4. The process of claim 1, further comprising: in response todetermining that the method is not frequently executed and the secondcode cache is not full, the computer processor placing metadata of thecompiled code in the second code cache.
 5. The process of claim 1,further comprising: the computer processor determining whether a numberof recompilations due to a heuristic exceeds a predetermined number,wherein the heuristic is used by the just-in-time compiler to determinewhether the method is frequently executed; and in response todetermining that the number of recompilations due to the heuristicexceeds the predetermined number, the computer processor adjusting theheuristic.
 6. The process of claim 1, further comprising: in response todetermining that the number of recompilations due to the method exceedsthe predetermined number, the computer processor placing the compiledcode of the method in the first code cache.
 7. A computer programproduct for efficient code cache management implemented by ajust-in-time compiler, the computer program product comprising acomputer readable storage medium having program code embodied therewith,the program code executable to: determine whether a method is frequentlyexecuted; in response to determining that the method is frequentlyexecuted, place a compiled code of the method in a first code cachewhich is a permanent code cache; determine whether a number ofrecompilations due to the method exceeds a predetermined number; inresponse to determining that the method is not frequently executed anddetermining that the number of recompilations due to the method does notexceed the predetermined number, determine whether a second code cacheis full, wherein the second code cache is a transient code cache; inresponse to determining that the second code cache is not full, placethe compiled code in the second code cache; and in response todetermining that the second code cache is full, flush a subdivision ofthe second code cache and then place the compiled code in the secondcode cache.
 8. The computer program product of claim 7, furthercomprising the program code executable to: in response to determiningthat the method is frequently executed, place metadata of the compiledcode in the first code cache.
 9. The computer program product of claim7, further comprising the program code executable to: in response todetermining that the method is not frequently executed and the secondcode cache is full, flush metadata in the subdivision of the second codecache.
 10. The computer program product of claim 7, further comprisingthe program code executable to: in response to determining that themethod is not frequently executed and the second code cache is not full,place metadata of the compiled code in the second code cache.
 11. Thecomputer program product of claim 7, further comprising the program codeexecutable to: determine whether a number of recompilations due to aheuristic exceeds a predetermined number, wherein the heuristic is usedby the just-in-time compiler to determine whether the method isfrequently executed; and in response to determining that the number ofrecompilations due to the heuristic exceeds the predetermined number,adjust the heuristic.
 12. The computer program product of claim 7,further comprising the program code executable to: in response todetermining that the number of recompilations due to the method exceedsthe predetermined number, place the compiled code of the method in thefirst code cache.
 13. A computer system for efficient code cachemanagement implemented by a just-in-time compiler, the computer systemcomprising: one or more processors, one or more computer readabletangible storage devices, and program instructions stored on at leastone of the one or more computer readable tangible storage devices forexecution by at least one of the one or more processors, the programinstructions executable to: determine whether a method is frequentlyexecuted; in response to determining that the method is frequentlyexecuted, place a compiled code of the method in a first code cachewhich is a permanent code cache; determine whether a number ofrecompilations due to the method exceeds a predetermined number; inresponse to determining that the method is not frequently executed anddetermining that the number of recompilations due to the method does notexceed the predetermined number, determine whether a second code cacheis full, wherein the second code cache is a transient code cache; inresponse to determining that the second code cache is not full, placethe compiled code in the second code cache; and in response todetermining that the second code cache is full, flush a subdivision ofthe second code cache and then place the compiled code in the secondcode cache.
 14. The computer system of claim 13, further comprising theprogram instructions executable to: in response to determining that themethod is frequently executed, place metadata of the compiled code inthe first code cache.
 15. The computer system of claim 13, furthercomprising the program instructions executable to: in response todetermining that the method is not frequently executed and the secondcode cache is full, flush metadata in the subdivision of the second codecache.
 16. The computer system of claim 13, further comprising theprogram instructions executable to: in response to determining that themethod is not frequently executed and the second code cache is not full,place metadata of the compiled code in the second code cache.
 17. Thecomputer system of claim 13, further comprising the program instructionsexecutable to: determine whether a number of recompilations due to aheuristic exceeds a predetermined number, wherein the heuristic is usedby the just-in-time compiler to determine whether the method isfrequently executed; and in response to determining that the number ofrecompilations due to the heuristic exceeds the predetermined number,adjust the heuristic.
 18. The computer system of claim 13, furthercomprising the program instructions executable to: in response todetermining that the number of recompilations due to the method exceedsthe predetermined number, place the compiled code of the method in thefirst code cache.